Information processing apparatus, control method, and storage medium storing program

ABSTRACT

The apparatus draws virtual objects as an image from a predetermined point of view, determines whether the virtual objects interfere with each other, calculates the region of interference of the virtual objects determined as interfering with each other, and outputs an image in which the region of interference located behind of the virtual objects as seen from the point of view is drawn.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an information processing apparatus fordrawing and displaying a virtual space image, a control method, and astorage medium storing a program.

Description of the Related Art

Recently, in the field of engineering and manufacturing, a reduction inperiod and cost of evaluation using a prototype has been demanded. Amixed reality (MR) system has been introduced for evaluating the ease ofassembly and maintenance by means of engineering data such as shape anddesign created on a CAD (computer-aided design).

One example of the items of an evaluation made by using a mixed realitysystem is a check as to whether or not it is possible to perform workwithout interference of a tool with any part other than the target part.In a mixed reality system for such an evaluation, when a person whotries out the system moves a virtual object (e.g., tool) to causeinterference with another virtual object (e.g., part), a function ofoffering the visual presentation of the region of interference in ahighlighted manner is sometimes demanded. A known simple way to providesuch highlighted presentation is to change the display color of theregion of interference into a highlight color that has been set inadvance. However, in this known method, there is problem that it is notpossible to visually present the region of interference in a case wherethe region of interference is hidden behind a virtual object that blocksthe line of sight between the eyepoint of the person trying out thesystem and the region of interference (line-of-sight blocking object).

To solve this problem, a method of, with a change in the display colorof an interfering virtual object in its entirety, generating an image ofthe virtual object is disclosed in Patent Publication No. 4756899. Inthis method, different highlight colors have been set in advance forvertices of a virtual object, and, in the event of the occurrence ofinterference, the region of interference is visually presented bychanging the color of the virtual object in its entirety into thehighlight color preset for the vertex corresponding to the region ofinterference. Therefore, even in a case where the region of interferenceis invisible from the person trying out the system, as long as at leasta part of the interfering virtual object is visible, it is possible torecognize the region of interference on the basis of the relationbetween each vertex and the corresponding highlight color.

In a mixed reality system, the following situation can be considered:virtual objects interfere with each other, and a part or a whole of theregion of interference is hidden behind a line-of-sight blocking objectand is therefore invisible from the eyepoint of a person who tries outthe system.

In such a situation, with a change in the display color of the region ofinterference into the preset highlight color alone, it is difficult tooffer the visual presentation of the region of interest to the persontrying out the system.

The technique disclosed in Patent Publication No. 4756899 makes itpossible to recognize the region of interference even if the region ofinterference is hidden, achieved by, as long as a part of the virtualobject is visible, changing the display color of the virtual object inits entirety. However, nothing is disclosed in this publication aboutvisualization of the region of interference itself. Moreover, if no partof the virtual object is visible, it is not possible to visually presentthe region of interest.

SUMMARY OF THE INVENTION

An information processing apparatus according to some aspect of thepresent invention comprises: a virtual object drawer configured to drawvirtual objects as an image seen from a predetermined point of view; aninterference determiner configured to determine whether the virtualobjects drawn by the virtual object drawer interfere with each other;

-   an interference region deriver configured to derive a region of    interference of the virtual objects determined as interfering with    each other by the interference determiner; and-   an image outputter configured to output an image in which the region    of interference located behind of the virtual objects as seen from    the point of view is drawn.

With these features disclosed in detail in this specification, it ispossible to offer the visual presentation of the region of interferenceeven when there exists any virtual object that blocks the line of sightbetween the eyepoint of a person who tries out a system and the regionof interference.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates the functional configurationof an information processing apparatus according to a first embodiment.

FIG. 2 is a flowchart that illustrates a series of processing of aninformation processing apparatus according to the first embodiment.

FIG. 3 is a detailed flowchart of interference region drawing processingaccording to the first embodiment.

FIG. 4A is a schematic view of an example of a virtual space image.

FIG. 4B is a schematic view of an example of a virtual space image.

FIG. 4C is a schematic view of an example of a virtual space image.

FIG. 4D is a schematic view of an example of a virtual space image.

FIG. 5 is a schematic view of an example of a virtual space image.

FIG. 6 is a schematic view of an example of a virtual space image.

FIG. 7 is a detailed flowchart of interference region drawing processingaccording to a second embodiment.

FIG. 8A is a schematic view of an example of a virtual space image.

FIG. 8B is a schematic view of an example of a virtual space image.

FIG. 9 is a detailed flowchart of interference region drawing processingaccording to a third embodiment.

FIG. 10A is a schematic view of an example of a virtual space image.

FIG. 10B is a schematic view of an example of a virtual space image.

FIG. 11 is a block diagram that illustrates an example of hardwareconfiguration for realizing an information processing apparatusaccording to an exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

With reference to the accompanying drawings, exemplary embodiments ofthe present invention will now be explained.

First Embodiment

FIG. 1 is a block diagram that illustrates the functional configurationof an information processing apparatus according to a first embodiment.

An information processing apparatus according to the present embodimentdraws a virtual space image, synthesizes the virtual space image with areal space image captured with a camera 107, and displays the result ofsynthesis as mixed reality video on a display unit 110.

A virtual object database 101 is a database in which virtual objectmodels are stored. The virtual object model includes, for example,shape, display color, and initial position and orientation.

The camera 107 acquires an image of a real space by shooting, and inputsit into the information processing apparatus.

A position-and-orientation input unit 108 estimates the position andorientation of the camera 107 in the real space on the basis of the realspace image captured with the camera 107. In the present embodiment, itis assumed that the position and orientation of the camera 107 coincideswith the position and orientation of the eyepoint of a person who triesout the system. Though not illustrated in the drawings, in the realspace, there exist markers placed for determining the position andorientation of virtual objects, and the person trying out the system canchange the position and orientation of the virtual objects by moving themarkers. When the present invention is put into practical application,the method of determining the position and orientation of the eyepointof the person trying out the system is not limited to the methoddescribed here. The same holds true for the position and orientation ofvirtual objects.

On the basis of the position and orientation of the eyepoint of theperson trying out the system and the position and orientation of thevirtual objects, which are inputted from the position-and-orientationinput unit 108, a virtual object position-and-orientation updating unit102 place the virtual eyepoint and the virtual objects to construct avirtual space. In the present embodiment, the shape of a virtual objectis expressed as a set of three-dimensional vertices and a set oftriangular polygons connecting the vertices. Therefore, the virtualobject position-and-orientation updating unit 102 converts thecoordinate value of each of the vertices of each of the virtual objectsinto a coordinate value in a three-dimensional coordinate system thathas the origin at the optical center of the camera 107.

A virtual object drawing unit 104 draws virtual objects as an imageviewed from the virtual eyepoint. When a three-dimensional virtualobject is drawn, hidden-line/hidden-surface removal processing isperformed. Hidden-line/hidden-surface removal is a technique forcorrectly expressing an anteroposterior relationship, for example,between virtual objects as to which one of the two is in front of theother, by not drawing lines and surfaces (faces) that are supposed to beinvisible.

An interference determination unit 103 determines whether virtualobjects interfere with each other (or one another) or not on the basisof the position and orientation of the virtual objects and the shapethereof. To determine whether two virtual objects interfere with eachother or not, the following procedure is taken. First, one triangularpolygon is taken out of each of the virtual objects. It is determinedwhether the two triangular polygons intersect with each other or not.This processing is performed for all combinations of triangularpolygons. If there is at least one intersecting combination, it isdetermined that the two virtual objects interfere with each other. Thisprocedure is repeated for all combinations of all virtual objects. Whenthe present invention is put into practical application, the method ofinterference determination is not limited to the method described here.

An interference region calculation unit 105 calculates the region ofinterference of the virtual objects determined as interfering with eachother by the interference determination unit 103. In the presentembodiment, the region of interference is expressed as a set ofintersecting triangular polygons and a set of lines of intersection.When the present invention is put into practical application, the methodof calculating the region of interference is not limited to the methoddescribed here.

An interference region drawing unit 106 draws, in a superposed manner,the region of interference calculated by the interference regioncalculation unit 105 on the image drawn by the virtual object drawingunit 104. When the interference region drawing processing is performed,all regions of interference are drawn irrespective of an anteroposteriorrelationship with the virtual objects by disabling thehidden-line/hidden-surface removal processing. That is, as viewed fromthe virtual eyepoint, the region of interference is drawn in front ofthe virtual objects drawn by the virtual object drawing unit 104.

In the present embodiment, as the method of drawing the region ofinterference, a set of lines of intersection is drawn in a preset linestyle. The preset line style means color, line width, and pattern (linetype), etc. For example, the line of intersection of the region ofinterference of virtual objects is rendered in red bold forhighlighting. This is called as highlight display of the region ofinterference. With the highlight display, the person trying out thesystem can visually recognize the region of interference.

A camera image synthesis unit 109 synthesizes the virtual space imagedrawn by the virtual object drawing unit 104 and the interference regiondrawing unit 106 with the real space image captured with the camera 107.The display unit 110 is a video see-through HMD (Head Mounted Display)that can perform stereoscopic 3D display. An HMD is provided with a pairof cameras and a pair of displays, one for the left eye and the otherfor the right eye. Therefore, the camera 107, the virtual object drawingunit 104, the interference region drawing unit 106, and the camera imagesynthesis unit 109 produce a left-eye image and a right-eye image. Thedisplay unit 110 is not limited to an HMD.

In the present embodiment, a mixed reality system that synthesizes avirtual space image with a real space image to produce mixed realityvideo is taken as an example. However, the scope of the presentinvention is not limited to the disclosed system. The present inventionmay be applied to other information processing apparatus that draws avirtual space image.

A series of processing of an information processing apparatus accordingto the first embodiment is illustrated in FIG. 2. An example of avirtual space image is illustrated in FIG. 4. In the followingdescription, as illustrated in FIG. 4A, a case where a box-shapedvirtual object 401 and a column-shaped virtual object 402 are arrangedis taken as an example.

In a step S201, the information processing apparatus reads the models ofthe virtual objects 401 and 402 out of the virtual object database 101.

In a step S202, the information processing apparatus receives an inputof a real space image captured with the camera 107.

In a step S203, the virtual object position-and-orientation updatingunit 102 reflects marker information onto the position and orientationof the virtual eyepoint and the position and orientation of the virtualobjects 401 and 402, wherein the marker information is inputted from theposition-and-orientation input unit 108 for determining the position andorientation of the camera 107 and the position and orientation of thevirtual objects 401 and 402.

In a step S204, the virtual object drawing unit 104 enablesdetermination using a depth buffer (Z buffer), which is commonly usedhidden-line/hidden-surface removal processing. In a step S205, thevirtual object drawing unit 104 draws the virtual objects 401 and 402.Therefore, when the processing of drawing the virtual objects 401 and402 is performed, it is determined on a pixel-by-pixel basis whether thedepth value of the virtual object 401, 402 on a pixel is less than thevalue stored for the pixel in the depth buffer or not, and the virtualobject 401, 402 is drawn on the pixel only in a case where said virtualdepth value is less than said stored value. In addition, the depth valueof the pixel in the depth buffer is updated. As a result of theprocessing described above, an image corresponding to FIG. 4A is drawn.When the present invention is put into practical application, thehidden-line/hidden-surface removal processing is not limited todetermination using a depth buffer.

In the present embodiment, the virtual eyepoint in the virtual spacecoincides with the position and orientation of the camera 107 in thereal space. This is because it is assumed that the position andorientation of the eyepoint of the person trying out the system in thereal space coincides with the position and orientation of the camera107.

In the real space, there exist markers placed for determining theposition and orientation of the virtual objects 401 and 402respectively, and the person trying out the system can change theposition and orientation of the virtual objects 401 and 402 by movingthe markers.

In a step S206, the interference determination unit 103 determineswhether the virtual objects 401 and 402 interfere with each other or noton the basis of the position and orientation of the virtual objects 401and 402 and the shape thereof updated by the virtual objectposition-and-orientation updating unit 102. If the two interfere witheach other, the process proceeds to a step S208. If not, the processproceeds to a step S207. FIG. 4B illustrates a state of interference inwhich the virtual object 402 interferes with the virtual object 401 as aresult of the movement of the marker corresponding to the virtual object402 by the person trying out the system from the state illustrated inFIG. 4A.

In the step S208, the interference region calculation unit 105calculates the region of interference of the virtual objects 401 and 402determined as interfering with each other by the interferencedetermination unit 103.

In a step S209, the interference region drawing unit 106 performsinterference region drawing processing. After that, the process proceedsto the step S207.

With reference to FIG. 3, the interference region drawing processingperformed in the step S209 will now be explained in detail.

In a step S301, the hidden-line/hidden-surface removal processing usingthe depth buffer is disabled. Therefore, in graphics drawn in subsequentsteps, all lines and surfaces are drawn in a superposed manner on theimage that has already been drawn, without taking the anteroposteriorrelationship into consideration.

In a step S302, the processing of rendering the line of intersection ofthe region of interference (borderline) in bold and preset color isapplied to the image drawn in the step S205. As a result of thisprocessing, a superposition-processed image with highlight display 403of the region of interference on the virtual objects 401 and 402 isgenerated as illustrated in FIG. 4B. In this image, the lower part ofthe virtual object 402 is not drawn because of thehidden-line/hidden-surface removal processing, in which itsanteroposterior relationship with the virtual object 401 is taken intoconsideration. In contrast, in the highlight display 403 of the regionof interference, all lines are drawn, including the part (upper half)that would be otherwise hidden by the virtual object 402. By this means,it is possible to offer the visual presentation of the region ofinterference, and the person trying out the system can easily understandthe region of interference.

Referring back to FIG. 2, in the step S207, the camera image synthesisunit 109 synthesizes, with the real space image captured with the camera107, the image of the virtual objects 401 and 402 together with thehighlight display 403 of the region of interference. The synthesizedimage is displayed on the display unit 110. After that, the processreturns to the step S202, and the series of processing in the subsequentsteps is repeated. In this way, it is possible to present, to the persontrying out the system, mixed reality video based on successive executionof real space shooting and superposition of a virtual space image on acamera-shot image.

In the present embodiment, an HMD that can perform stereoscopic displayis used as the display unit 110. Therefore, the steps S202 to S209 areexecuted for each of the left eye and the right eye. This makes itpossible for the person trying out the system with the wearable HMD toenjoy 3D mixed reality video based on binocular disparity. When thehighlight display 403 of the region of interference is performed, thehidden-line/hidden-surface removal processing using the depth buffer isdisabled. For this reason, with the 2D image alone, without a hint bythe line/surface that would be otherwise hidden, there is a possibilitythat it might be difficult to understand the anteroposteriorrelationship with the virtual object. This is why an HMD, in which,because of the use of stereoscopic 3D vision, depth perception based onbinocular parallax provides a clue to the understanding of theanteroposterior relationship, is suitable.

Next, with reference to FIG. 4C, a state in which there exists a virtualobject that blocks the line of sight between the eyepoint of the persontrying out the system and the region of interference (line-of-sightblocking object) will now be explained. In the example described below,the line-of-sight blocking object 404 is a virtual model of the hand ofa person who tries out the system. Since the line-of-sight blockingobject 404 is also a virtual object, it is drawn in the step S205 of theflowchart in FIG. 2 similarly to the virtual objects 401 and 402. Thatis, the hidden-line/hidden-surface removal processing using the depthbuffer is enabled and, therefore, a part of the virtual objects 401 and402, or more specifically, the part behind the line-of-sight blockingobject 404, is not drawn. On the other hand, since thehidden-line/hidden-surface removal processing is disabled for thehighlight display 403 of the region of interference of the virtualobjects 401 and 402, it is in front of all of the virtual objects. Bythis means, it is possible to offer the visual presentation of theregion of interference as illustrated in FIG. 4C, and the person tryingout the system can easily understand the region of interference.

Next, with reference to FIG. 4D, a state of GUI-superposed display(display with graphical user interface) that is performed if necessarywhen the image generated by the camera image synthesis unit 109 isdisplayed on the display unit 110 will now be explained. As illustratedin FIG. 4D, it is assumed here that there is an overlap between thedisplay area of GUI 405 and the area of the highlight display 403 of theregion of interference. The GUI 405 is displayed in front of thehighlight display 403 of the region of interference, which is in frontof the virtual objects. This can be realized by displaying the GUI 405in a superposed manner on the image generated by the camera imagesynthesis unit 109. When the present invention is put into practicalapplication, the information displayed at the frontmost position is notlimited to GUI. When information that is of greater importance to aperson who tries out the system than virtual objects is displayed on thescreen, the frontmost information display can be obtained by superposingthe information on the image generated by the camera image synthesisunit 109.

FIG. 11 is a block diagram that illustrates an example of hardwareconfiguration for realizing an information processing apparatusaccording to the present embodiment. In FIG. 11, the same referencenumerals are assigned to components that are the same as those ofFIG. 1. They are not explained here.

The numeral 1101 denotes a CPU. The CPU 1101 controls the entireoperation of the apparatus. The numeral 1102 denotes a RAM. The RAM 1102is used as a workspace when the CPU 1101 performs processing whilecontrolling the components. The numeral 1103 denotes a ROM. A controlprogram, various application programs, and data, etc. are stored in theROM 1103. The CPU 1101 reads the control program out of the ROM 1103into the RAM 1102, and runs it, thereby configuring the virtual objectposition-and-orientation updating unit 102, the interferencedetermination unit 103, the virtual object drawing unit 104, theinterference region calculation unit 105, and the interference regiondrawing unit 106, etc. illustrated in FIG. 1. The numeral 1104 denotesan external memory. The virtual object database 101 is in the externalmemory 1104.

First Variation Example

In the first embodiment, the line of intersection of the region ofinterference is rendered in a preset line style (for example, in redbold). However, the method of highlight display is not limited to theforegoing example. For example, different line styles such as a linestyle for rendering a hidden line and another line style for renderingother line (non-hidden line) may be applied.

A method for highlight display 501 of the region of interferenceillustrated in FIG. 5, in which the hidden line is rendered as a dottedline, and the non-hidden line as a solid line, will now be explained.The illustrated modified highlight display is realized by enabling thefollowing processing instead of disabling the hidden-line/hidden-surfaceremoval processing using the depth buffer in the step S301.

When the processing of highlight display of the region of interferenceis performed, it is determined on a pixel-by-pixel basis whether thedepth value of the virtual object on a pixel is less than the valuestored for the pixel in the depth buffer (meaning that the pixel is apart of the non-hidden line) or not, and a first style is applied to thepixel if said virtual depth value is less than said stored value. If not(meaning that the pixel is a part of the hidden line), a second style isapplied to the pixel. As a result of this processing, the hidden lineand the non-hidden line are rendered in different styles as in thehighlight display 501 for the virtual objects. The term “style” as usedherein means a combination of color, line width, and pattern (linetype), etc., and the first and second styles have been set in advance.

The rendering of a hidden line and a non-hidden line in different stylesas in the above example provides a clue to the understanding of ananteroposterior relationship. It can be expected that this will make theregion of interference easier to recognize.

Second Variation Example

In the first embodiment, the line of intersection of the region ofinterference is rendered in a preset line style (for example, in redbold). However, the method of highlight display is not limited to theforegoing example. For example, in addition to, or instead of, therendering of the line of intersection of the region of interference inthe form of a line or lines, it may be rendered in the form of a surfacearea. As in highlight display 601 in FIG. 6, with the highlighting ofthe surface area encircled by the line of intersection in addition tothe highlighting of the line of intersection, it can be expected thatthe region of interference will be easier to notice.

Instead of presetting the line/surface style (color, line width,pattern, etc.) for highlight display, it may be determined dynamicallyon the basis of the display color or pattern of the virtual objectsinterfering with each other. The only thing necessary for thismodification is to add a highlight display style determination unitbetween the interference determination unit 103 and the interferenceregion drawing unit 106. The highlight display style determination unitdetermines, for example, the complementary color of the display color ofthe virtual object as the color of highlight display. It can be expectedthat this will make the difference between the display color of thevirtual object and the color of the highlight display of the region ofinterference clearer.

Second Embodiment

A second embodiment will now be explained. The same configuration andprocessing as those of the first embodiment are not explained in detailhere. The focus of the description will be on the point of differencefrom the first embodiment.

In the first embodiment, highlight display is performed for the entireregion of interference calculated by the interference region calculationunit 105. However, the scope of the present invention is not limited tothe foregoing example. In the description below, a state illustrated inFIG. 8A is taken as an example. There are two virtual objects 402 and801 interfering with the virtual object 401. The virtual object 801 is“included” in, that is, located completely inside, the virtual object402. For easier understanding, in FIG. 8A, all hidden lines of thevirtual objects 401, 402, and 801 are shown as dotted lines.

In FIG. 8A, for highlight display for all regions of interferencesimilarly to highlight display for the entire region of interference inthe first embodiment, highlight display 403 and highlight display 802are on the screen. However, since the virtual object 801 is included inthe virtual object 402, all of the lines and surfaces of the virtualobject 801 are hidden lines and hidden surfaces. Therefore, they are notdrawn by the virtual object drawing unit 104. For this reason, thehighlight display 802 of the region of interference only is visuallypresented to the person trying out the system. Such highlight-onlydisplay might make the person trying out the system feel somethingwrong.

In the present embodiment, an example of reducing such a feeling ofstrangeness by presenting the highlight display 403 only to the persontrying out the system, without the highlight display 802 of the regionof interference, is described.

The strangeness reduction can be realized by replacing the interferenceregion drawing processing in the step S209 of the flowchart in FIG. 2with the flowchart in FIG. 7.

The processing performed in a step S701 is the same as that of the stepS301.

In a step S702, the interference region calculation unit 105 performsinclusion determination processing for the calculated region ofinterference. In the present embodiment, the region of interference isexpressed as a set of triangular polygons and a set of lines ofintersection. The inclusion determination processing is performed foreach triangular polygon of the region of interference. For the purposeof judging whether a certain triangular polygon is included in a virtualobject or not, AABB (Axis-Aligned Bounding Box) that approximates theshape of the virtual object is used. AABB is a box whose edges areparallel to the X, Y, and Z axes in a three-dimensional space, with thesmallest measure within which all of the vertices of a virtual objectlie. In a case where all of the vertices of a certain triangular polygonlie within the AABB of a certain virtual object, it is determined thatthe triangular polygon is included in the virtual object. When thepresent invention is put into practical application, the method ofinclusion determination is not limited to the method described here.

In a step S703, on the basis of the result of inclusion determination inthe step S702, the highlight display of the region of interference isperformed if a certain triangular polygon is not included in any of thevirtual objects determined as interfering with each other by theinterference determination unit 103, or the highlight display of theregion of interference is not performed if the triangular polygon isincluded in at least one of these interfering virtual objects.

With the above processing, it is possible to eliminate the highlightdisplay 802 of the region of interference for the virtual object 801,which is included in the virtual object 402, as illustrated in FIG. 8B.For easier understanding, in FIG. 8B, all hidden lines of the virtualobject 801 are shown as dotted lines.

Third Embodiment

A third embodiment will now be explained. The same configuration andprocessing as those of the first embodiment are not explained in detailhere. The focus of the description will be on the point of differencefrom the first embodiment.

In the first embodiment, line-based rendering or surface-area-basedrendering is applied to the line of intersection. However, the scope ofthe present invention is not limited to the foregoing example. Forexample, as illustrated in FIG. 10A, non-transparent rendering orsemi-transparent rendering may be used for visualizing the shape of theregion of interference.

In the present embodiment, this can be realized by replacing theinterference region drawing processing in the step S209 of the flowchartin FIG. 2 with the flowchart in FIG. 9.

In a step S901, as illustrated in FIG. 10B, an image 1001 forvisualizing the shape of the region of interference calculated by theinterference region calculation unit 105 (hereinafter referred to as“interference region image”) is generated. Since thehidden-line/hidden-surface removal processing enabled in the step S204remains valid in this step, the interference region image 1001 generatedhere is a “hidden-processed” image.

In a step S902, the hidden-line/hidden-surface removal processing isdisabled.

In a step S903, the interference region image 1001 generated in the stepS903 is superposed in a non-transparent state or in a semi-transparentstate on the image of the virtual objects drawn in the step S205.

As a result of the above processing, the shape of the region ofinterference is visualized non-transparently or semi-transparently asillustrated in FIG. 10A. By this means, it is possible to offer thevisual presentation of the region of interference, with thevisualization of its shape, and the person trying out the system caneasily understand the region of interference.

In the present embodiment, instead of presetting the style of theinterference region image 1001, it may be determined dynamically on thebasis of the display color or pattern of the virtual objects interferingwith each other.

Though the specific concept of the present invention has been describedwith exemplary embodiments, the embodiments merely illustrate examplesfor implementing the present invention, and the technical scope of thepresent invention shall not be construed limitedly from these examples.That is, the present invention can be implemented in a variety of modeswithout departing from its technical spirit or principal features.

For example, highlight display for highlighting the line of intersectionof the region of interference described in the first or secondembodiment may be combined with the interference region image of thethird embodiment.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2015-135540, filed Jul. 6, 2015, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus, comprising:a memory storing instructions; and a processor which is capable ofexecuting the instructions causing the information processing apparatusto: draw virtual objects as an image seen from a viewpoint correspondingto a set of position and orientation of a head mounted display held by auser; determine whether a first object and a second object of the drawnvirtual objects collide with each other; derive, in a case where it isdetermined that the first object and the second object collide with eachother, a contour line formed by collision of outer surfaces of the firstobject and the second object; draw the contour line, the drawn contourline including a part hidden from the viewpoint by the first object orthe second object; generate a display image by combining the drawnvirtual objects, the drawn contour line, and a captured image of aphysical space where the user is, in a case where, seen from theviewpoint, a virtual hand object which represents a hand of the userhides the collision of the first object and the second object, the firstobject, the second object, and the virtual hand object, are displayed,in back of the drawn contour line, and a virtual message object whichrepresents message to the user is displayed in front of the drawncontour line, in the display image; and output the display image to thehead mounted display.
 2. The information processing apparatus accordingto claim 1, wherein the contour line is drawn without applyinghidden-line removal processing and/or hidden-surface removal processingthereto, and wherein said hidden-line removal processing and/orhidden-surface removal processing is processing to not draw one or moreof the virtual objects that are invisible from the viewpoint.
 3. Theinformation processing apparatus according to claim 1, wherein thecontour line is drawn by using a line or using a surface, or both. 4.The information processing apparatus according to claim 3, wherein theprocessor capable of executing the instructions further causes theinformation processing apparatus to: determine a style of the line orthe surface, or both, used for drawing the contour line, depending onproperty of the virtual objects determined as colliding with each other.5. The information processing apparatus according to claim 4, whereinthe processor capable of executing the instructions further causes theinformation processing apparatus to determine a color that is differentfrom a color of the virtual objects determined as colliding with eachother as a color of the line or the surface, or both, used for drawingthe contour line.
 6. The information processing apparatus according toclaim 1, wherein the processor capable of executing the instructionsfurther causes the information processing apparatus to draw a line ofintersection of the contour line by applying different line styles. 7.The information processing apparatus according to claim 6, wherein thedifferent line styles comprise a first line style for rendering a hiddenline and a second line style for rendering another line thereto.
 8. Theinformation processing apparatus according to claim 1, wherein, when itis determined that the second object of the virtual objects and a thirdobject of the virtual objects collide with the first object of thevirtual objects and, when it is determined that the third virtual objectis included in the second virtual object, the line formed by collisionof outer surfaces of the first virtual object and the third virtualobject is not drawn.
 9. The information processing apparatus accordingto claim 1, wherein the image outputted is obtained by synthesizing ashot image of a physical space with the virtual objects.
 10. Theinformation processing apparatus according to claim 1, wherein the headmounted display is wearable on a head of a user.
 11. A method forcontrolling an information processing apparatus, comprising: virtualobject drawing, in which virtual objects are drawn as an image seen froma predetermined viewpoint corresponding to a set of position andorientation of a head mounted display held by a user; interferencedetermination, in which it is determined whether a first object and asecond object of the virtual objects drawn in the virtual object drawingcollide with each other; contour line deriving, in a case where it isdetermined that the first object and the second object collide with eachother, in which a contour line formed by collision of outer surfaces ofthe first object and the second objects is derived; contour linedrawing, in which the contour line is drawn, the drawn contour lineincluding a part hidden from the viewpoint by the first object or thesecond object; display image generating, in which a display image isgenerated by combining the drawn virtual objects, the drawn contourline, and a captured image of a physical space where the user is, in acase where, seen from the viewpoint, a virtual hand object whichrepresents a hand of the user hides the collision of the first objectand the second object, the first object, the second object, and thevirtual hand object, are displayed, in back of the drawn contour line,and a virtual message object which represents message to the user isdisplayed in front of the drawn contour line, in the display image; andimage outputting, in which the display image is outputted to the headmounted display.
 12. A non-transitory computer-readable storage mediumstoring instructions that, when executed by a processor, causes acomputer to: draw virtual objects as an image seen from a predeterminedviewpoint corresponding to a set of position and orientation of a headmounted display held by a user; determine whether a first object and asecond object of the drawn virtual objects collide with each other;derive, in a case where it is determined that the first object and thesecond object collide with each other, a contour line formed bycollision of outer surfaces of the first object and the second object;draw the contour line, the drawn contour line including a part hiddenfrom the viewpoint by the first object or the second object; generate adisplay image by combining the drawn virtual objects, the drawn contourline, and a captured image of a physical space where the user is, in acase where, seen from the viewpoint, a virtual hand object whichrepresents a hand of the user hides the collision of the first objectand the second object, the first object, the second object, and thevirtual hand object, are displayed, in back of the drawn contour line,and a virtual message object which represents message to the user isdisplayed in front of the drawn contour line, in the display image; andoutput the display image to the head mounted display.